Audit Report No. 28 2008-09

Chapter 4 Quality and Integrity of the Department of Veterans’ Affairs Income Support Records

Introduction

4.1                   The Repatriation Commission was officially established on 1 July 1920 with the passing of the Australian Soldiers’ Repatriation Act 1920, to provide support for veterans, widows and their families.[1] This Act was replaced by the Veterans’ Entitlements Act 1986 (the Act), which retained the functions of the Repatriation Commission. Under the Act, the Commission delegates its powers to the Department of Veterans’ Affairs (DVA) to grant pensions and other benefits to veterans and their dependants, and certain other eligible people.

4.2                   The service pension provides regular income support for people with limited means of income and is broadly equivalent to the Centrelink age and disability pensions. However, it is payable five years earlier than the age pension in recognition of the effects of war. The two critical criteria required of claimants in the Act are to be a veteran and have rendered qualifying service. Age and residency requirements are also mandatory for some pensions. There are rules related to assets and income that also affect pension eligibility and pension amount.

4.3                   The total required to deliver this output in 2007-08 was $44,744,000.[2] DVA’s client population has declined over the past three financial years. At 30 June 2008, the income support pensioner population was 4.6 per cent less than was reported in the previous financial year. There has been a corresponding downwards trend in the department’s workload.[3]

4.4                   In order to better manage the services provided to Australia’s declining population of war veterans and the expected decrease in its workload and funding, DVA undertook a major restructure in 2005-06. This restructure, known as oneDVA, was the foundation for a new approach to the way DVA delivers its services to veterans. The reorganisation involved a move away from variable State-based practices and the expansion of geographically dispersed teams resourced along national business lines. The department’s strategy to standardise its operations across all service areas was complemented by the introduction of the first phase of a new ‘oneVoice’ telephone service environment, the Veterans’ Service Centre (VSC).

4.5                   The department’s administration of income support benefits relies on a combination of complex older heritage and new information technology (IT) systems. These systems contain extensive electronic records of personal and other information about DVA’s clients and are used to process income support claims and other related work. In February 2008, the department’s client databases held 1,580,546 total client records, of which 264,248 involved an income support payment.

4.6                   In parallel with DVA’s restructure, the department also recognised the need to modernise its IT and reduce its expenditure on maintenance of heritage systems.[4] As part of this initiative, the department purchased Cúram, an off-the-shelf, integrated IT application framework designed for service delivery environments. This IT-enabled business change offered the potential for better functionality of DVA’s IT systems based on up-to-date business rules, more reliable information underpinning decision-making and reporting, and increased convenience for veterans.

4.7                   At the time of the audit DVA was taking a phased approach to implementing Cúram, with data migration (from the department’s heritage IT systems into Cúram) scheduled over a number of years. The initial phase of the department’s modernisation of its IT systems occurred in 2006, when several projects were implemented. This involved the migration of over 1.5 million records containing client personal information from heritage databases into the new Cúram environment. Despite DVA’s considerable project planning, data testing and cleansing to prepare the heritage data for transfer into Cúram, unanticipated data incompatibility and integration issues emerged during the migration of the data. To enable the transfer of the data into Cúram, heritage records with blank date of birth fields were populated by DVA with ‘dummy’ data.[5]

4.8                   The department has a major initiative to clean up data integrity errors. This is the Data Integrity Cleanup Exercise (DICE) project which predominantly revolves around correcting:

n  data errors that were transferred from heritage systems into Cúram; and

n  newly created problems that arose as a result of incompatibility issues between heritage and Cúram systems during data migration.

4.9                   The department also has an ongoing program for data cleanup – the Data Integrity Problems (DIPs) work. DIPs activity is directed to correcting complex data errors unable to be resolved directly via the existing applications.

4.10               The cost of implementing Cúram was to be offset by ongoing savings in administration and program costs arising from the improved IT framework and de-commissioning of relevant heritage systems.[6]

The Audit[7]

Audit scope and objective

4.11               The objective of the audit was to examine the quality and integrity of DVA’s income support records and to report on the effectiveness of the department’s management of the data and how it impacts on service delivery. The audit included an examination of:

n  DVA’s management of the data including the quality of data stored on its client databases, the processing of claims and payment processing;

n  the accuracy, completeness and reliability of DVA’s electronic income support records; and

n  the impacts the quality of data has on service delivery, and related issues of customer support and feedback.

4.12               The audit focused on data integrity issues associated with the various types of service pension, income support supplement, the social security age pension and other related allowances. The accuracy and completeness of records of selected mandatory fields and other key fields that underpin the integrity of DVA’s income support records were examined.

4.13               The ANAO’s data extraction and analysis encompassed 1,580,546 records in DVA’s production environment which included the records of 264,248 income support clients who were in payment.[8] While the audit did not directly examine the accuracy of individual payments, it examined underlying data integrity issues that can impact on the accuracy of payments, such as the current status of client asset and income information.

4.14               The audit also included a limited number of reviews of paper files and consideration of documentation associated with DVA’s IT governance, particularly in relation to data management.

Overall audit conclusion

4.15               The ANAO made the following overall audit conclusion:

The Department of Veterans’ Affairs (DVA) has been undergoing significant change since 2005-06. Following a review of its service delivery arrangements, DVA adopted a new business and information technology (IT) strategy, in recognition of the declining population of war veterans and expected 30-50 per cent decrease in the department’s workload over the next 10 years. A key element of DVA’s IT strategy is to reduce expenditure on the maintenance of IT systems over time, by eventually decommissioning its heritage systems. The purchase of Cúram, an off-the-shelf IT product designed for social welfare environments, was a key part of this strategy. DVA identified Cúram as an enabler for its new model of business operation, known as oneDVA.

A challenge for DVA in this environment is balancing the resources required to maintain its heritage IT systems relative to its investment in new IT capability with its greater functionality. While Cúram is designed to be the ‘source of truth’ for client personal data and to provide a platform to better manage DVA’s data in the future, it is still in the early stages of implementation. Income support data is not yet scheduled for migration into Cúram, and DVA continues to be dependant on the integrity of the data stored in heritage systems for administering income support payments. In this environment, the department relies heavily upon the corporate knowledge held by a few key staff about its IT systems and business processes.

Overall, the poor quality of the data in DVA’s electronic databases is affecting the efficiency and reliability of the department’s decision-making, and its internal and external reporting. While in most cases there was sufficient evidence in DVA’s multiple systems and the hard copy customer records examined to support its clients’ eligibility for income support benefits, the audit revealed:

n  the department’s management of electronic data and data integrity issues was not effective;

n  key fields in many electronic records were not accurate, complete or reliable; and

n  inaccurate recording and reporting of complaints and compliments in the department’s Feedback Management System.[9]

Management of data and data integrity issues

4.16               The ANAO found numerous data integrity problems during the audit, including:

n  DVA clients with more than one Unique Identification Number (UIN) continuing to increase since the previous audit of DVA’s administration of Repatriation Health Cards in 2003-04;

n   DVA not having a complete electronic record of the qualifying service details for 41 per cent of veterans eligible for the age service pension;

n  exempt assets of clients being disregarded beyond the legislated exemption period for the purpose of the assets test;

n  fragmentation of client information across multiple records; and

n  cases of pension misclassification, requiring DVA to further analyse the raw data to ensure the information reported was meaningful and reliable.

4.17               The ANAO found that these and other data integrity issues identified by the audit increase the risk of DVA providing untimely advice and incorrect payments, and reduce the department’s capacity to provide assurance that the right person is receiving their correct entitlement. To mitigate this risk, the department has a range of administrative processes and checks in place. However, these add to the costs of administering the program. This situation also limits the ability of the department to garnish the dividends of oneDVA.

4.18               According to the ANAO, the quality of DVA’s data would be substantially improved through the development and deployment of an organisation-wide data integrity improvement strategy, underpinned by stronger governance arrangements. In recent years, DVA has reviewed both its IT governance and committee structure and identified similar issues to those reported by the ANAO in this audit, including a need for greater clarity across the department for the authority, ownership and control of data management and data integrity issues. Strengthening oversight arrangements would assist DVA to monitor the progress of the strategy, as well as align elements with the roll-out of Cúram and recognise the interdependencies within the oneDVA initiative. DVA’s progress would also be facilitated by setting targets and timeframes for reviewing records, and better utilising the opportunities presented by client and department initiated contacts, including compliance and review work, to improve data integrity.

Accuracy, completeness and reliability of key fields in client records

4.19               The ANAO found that DVA’s policy and procedural controls for new claims processing and updating of client records would benefit from review and consolidation. In particular, data entry controls and support materials should be standardised across the department’s State offices and the Veterans’ Service Centres (VSC), consistent with the oneDVA strategy. This would help to reduce data input errors and support consistency in decision-making and client records management generally. Additional assurance would be gained from a greater focus on data input standards and controls, and procedural compliance around claims processing and updating of client records.

4.20               Veterans/clients receiving the maximum pension or receiving a part pension (and considered by DVA to be low risk), are generally not reviewed through DVA’s review program measures unless pensioners notify of a change in their circumstances. This has potential service delivery impacts when cases are not reviewed for a number of years as clients could incur unexpected debts or by underpaid for a significant period. Retrospective adjustments applied to pensions over a number of years does create higher administrative costs for DVA, when having to account for multiple changes to a pensioner’s circumstances.

4.21               In 2004, DVA introduced its Enhanced Compliance Program (ECP) in order to manage the risk of pensioner non-compliance.[10] The ECP targets cases profiled by DVA as high risk with no recent review activity, or cases with potentially volatile income and assets. As well as achieving higher than expected benefit payment savings, the ECP has been successful in updating the current status of a client’s circumstances. However, this program, coupled with the program of two-yearly reviews of pensioners receiving less than the maximum pension rate, reviews less than 7 per cent of the total income support population. Overall, more than 70 per cent of clients have not had a review that updates all of their previously submitted asset information, for eight years or more.

4.22               DVA has a range of IT controls and assurance programs, including the ECP and client contact activities, which present an opportunity to improve data integrity. These compliance activities and other direct contacts with clients provide opportunities to implement cost-effective arrangements to validate or correct client information. Improved data integrity would provide greater assurance that DVA’s clients are receiving their correct income support entitlement and associated services.

Management of feedback data and service delivery impacts

4.23               The ANAO indicated that the unreliability of DVA’s client feedback data limited the department’s capacity to effectively utilise the intelligence gathered from complaints and compliments to assist in setting client service priorities and to systematically monitor and generate reliable public reports. DVA’s proposed new feedback management system was found to be in the early planning stages, with the projected timetable and resources yet to be defined. DVA advised that, in the meantime, its existing Feedback Management System (FMS) would continue to be used with its known data quality shortcomings.

4.24               The ANAO concluded that, in the interim, it would be prudent for DVA to raise departmental staff awareness of the need for all feedback to be recorded in the existing FMS and to ensure compliance with the department’s Procedural Policy-Handling Feedback from the Veteran Community. In doing so, the focus should be on awareness raising and highlighting the value DVA places on client feedback and the intelligence gathered from complaints, as a driver of improvements to business and service quality. This would assist in ensuring that client feedback is accurately recorded and appropriately managed day-to-day and ensure its availability for long term business and service delivery improvement purposes.[11]

ANAO recommendations

4.25               The ANAO made the following recommendations:

Table 4.1        ANAO Recommendations, Audit Report No. 28 2008-09

1.	To improve the governance of income support data, the ANAO recommends that the Department of Veterans’ Affairs: ·         clarifies the authority and role of its Data Integrity Sub-Committee responsible for resolving issues related to data ownership and enhancing the integrity of data held in the department’s databases; and ·         develops an agency-wide strategy, assigning ownership for data management and integrity issues to business areas. DVA response: Agreed
2.	The ANAO recommends that the Department of Veterans’ Affairs: ·         documents a controls framework for income support and systems based business rules; and ·         evaluates its current IT controls and assurance activities to determine which elements are most efficient and effective in improving data integrity. DVA response: Agreed
3.	To enhance the quality and integrity of income support records, the ANAO recommends that the Department of Veterans’ Affairs: ·         strengthens its data entry system and procedural controls by implementing a standard set of national procedures for income support client records management; ·         validates customer data when interacting with clients and during income support assurance activities; and ·         updates incorrect data in all key fields of active electronic client records prior to granting a payment or benefit. DVA response: Agreed
4.	The ANAO recommends that the Department of Veterans’ Affairs implements interim measures to enhance the quality of its feedback data while designing the new feedback system to manage stakeholder feedback by: ·         improving compliance by staff with the department’s current procedural policy; and ·         raising staff awareness of the need for all complaints and compliments data to be recorded in the department’s existing Feedback Management System. DVA response: Agreed

The Committee’s review

4.26               The Committee held a public hearing on Monday 15 June 2009, with the following witnesses:

n  Australian National Audit Office (ANAO);

n  Department of Veterans’ Affairs (DVA).

4.27               The Committee took evidence on the following issues:

n  current status of the Cúram project;

n  data integrity;

n  complaints;

n  update of clients’ asset records;

n  incomplete electronic records of service; and

n  risks associated with the reliance on corporate knowledge held by a few key staff members.

Current status of the Cúram project

4.28               The Department of Veterans’ Affairs reported to the Committee that since the Audit Report was presented the Cúram project has been put on hold, largely due to financial considerations. A review of the modernisation of the Department’s information technology systems is being undertaken to establish ‘which is the best platform and what is the best solution to our needs going forward’.[12]

4.29               DVA was anxious to stress to the Committee that the current difficulties with regard to the implementation of the modernisation of the agency’s information technology systems in no way impact on the correctness or accuracy of payments made to veterans. The General Manager, Support Division, was quick to point out that none of their accountability processes have been compromised:

… the current state of our systems in no way fetters our capacity to inform government or inform anybody else about the current state of DVA’s payments, benefits, expenditure and so forth.[13]

 

Recommendation 8
	The Committee recommends that the Department of Veterans’ Affairs (DVA) report back to the Committee at the conclusion of the review of the agency’s information technology systems and provide the Committee with details of the system chosen, the implementation plan and how the plan will address the Australian National Audit Office’s recommendations.

 

Data integrity

4.30               The ANAO identified links between the integrity of DVA’s data and its data entry controls and recommended that the agency take specific steps to mitigate the risk of errors and improve quality.[14] The recommendation calls for a standard set of national procedures, the validation of customer data whenever there is client contact and updating of incorrect client data before granting a payment or benefit.[15]

4.31               DVA reported that it has instigated a review of its procedures which was to be concluded in September 2009 and that identified changes will be implemented as soon as possible.  The Department maintains that it already validates data during client contact and that staff are continually being made aware of the need to follow this procedure. The National Manager, Income Support, assured the Committee that this is an ongoing process:

… we sent out a business line to all system users and client contact staff just last week reinforcing the message that they follow the Departmental protocols of updating all the client data information, which picks up gender, title, date of birth, name, address and contact details for the client.[16]

Complaints

4.32               The Committee expressed concern in the rise in complaints and drop in client satisfaction with complaints handling in 2006-07 identified in the Report. This is despite a reported overall high rate of client satisfaction with the Department’s services. Although they did not question the veracity of the findings, the ANAO detailed a number of weaknesses in the agency’s complaint handling procedures and reporting.

4.33               DVA explained that the total figures masked varying levels of satisfaction with the Department between clients covered under the Veterans Entitlement Act 1986 (VEA), the Military Rehabilitation and Compensation Act 2004 (MRCA) and the Safety, Rehabilitation and Compensation Act 1988 (SRCA). Clients under the VEA tend to be more satisfied with the service than those under the MRCA and SRCA. DVA provided a number of reasons for this higher dissatisfaction among the latter group:

It is a system where there is a greater emphasis on rehabilitation, they are younger individuals, they are people who have left the Australian Defence Force (ADF), which they had seen as potentially their career for life, and they are people who are clearly in a situation where they need a lot more work and assistance in getting them back to the position they were in prior to their injury and their discharge from the ADF.[17]

4.34               The Committee requested a disaggregation of the complaints data for 2006-07 and 2007-08 to allow a comparison to be made between both years. The disaggregated figures are available in Appendix D.

4.35               The Committee sought further information on the steps being taken to rectify the dissatisfaction being expressed by clients under the MRCA and SRCA. The Department explained that it has set up a younger veteran’s task force to examine the issues and develop solutions. DVA has identified communication channels as of particular concern to younger clients and is exploring the use of new technology to facilitate better communication between these clients and the Department. Examples of reforms that are improving service delivery to clients include:

n  a single claim form for all clients no matter which piece of legislation they are eligible under;

n   more efficient transfer of medical and service records from Defence to the Department; and

n  a single health examination form which reduces the need to consult separate doctors or practitioners.[18]

Update of clients’ asset records

4.36               The service pension for DVA clients is calculated using two tests: an income test and an asset test.[19] The ANAO found that while changes to income are monitored through regular reviews and data matching with Centrelink and the Australian Taxation Office, identifying changes to the value of assets is a more difficult task.[20] The audit report found that ‘more than 70 per cent of client’s asset records have not had all of their previously submitted asset items fully updated for eight years or more.’[21]

4.37               The Committee expressed concern that this backlog could be detrimentally affecting the income of veterans. DVA reported that in accordance with ANAO recommendations, they were undertaking a review of their data entry system and procedural controls, scheduled to be completed by September 2009.[22]

 

Recommendation 9
	That the Department of Veterans’ Affairs (DVA) provide the Joint Committee of Public Accounts and Audit (JCPAA) with a copy of the review of the DVA’s data entry system and procedural controls as soon as practicable after the tabling of the Committee’s report.

 

Incomplete electronic records of service

4.38               Qualifying service (QS) is the criteria used to establish a veteran’s eligibility for a service pension. The ANAO found that details of service are recorded on paper files but that 41 per cent of electronic records were incomplete with regard to these details.[23] As DVA advised that paper files are not used after the initial assessment, ANAO concluded that future decisions could be based on incomplete records which would increase the margin for error.[24]

4.39               DVA told the Committee that the transfer of information from paper records to the electronic database is resource intensive and not a high priority.[25] The Department explained that once qualifying service is established there is rarely any need to access the details:

… we do not consider the transfer of those to electronic files as being a priority because we just do not access those files once we have got the person in payment and eligible and we are confident of those particular details extracted from the paper files.[26]

4.40               The Committee nonetheless concurs with the ANAO that “when information is fragmented across different records, this leads to inefficient administration and increases the risk of error.”[27]  The Committee expects new entrants into the system with qualifying service to be entered into electronic records as a matter of course, and reiterates ANAO Recommendation No. 3, which calls for stronger data entry systems and procedural controls, better validation of customer data, and a commitment to updating incorrect data.

Corporate knowledge and key staff

4.41               Throughout the Audit Report the ANAO highlighted the reliance by the DVA on the corporate knowledge of a few key staff to both interpret raw data and negotiate its IT system.[28] The Committee expressed concern that this presents a considerable risk if this knowledge is lost as staff retire or leave the organisation and asked what the Department is doing to mitigate the risk.

4.42               DVA assured the Committee it has noted the Report’s finding and explained that the risk is chiefly to do with the heritage IT systems. As the information on these systems is migrated to newer systems, the heritage systems will be used less and less with a consequent decrease in the number of staff familiar with them.[29]  The issue is being considered in the review of ICT currently being undertaken by the Department.[30]

Conclusion

4.43               The Committee acknowledges that the ANAO found no instance of incorrect payments being made to DVA clients. It understands the importance of reassuring veterans that there is no reason for anxiety on their part with regard to the correctness or accuracy of their payments.

4.44               However, the Committee is concerned at the discrepancies and errors highlighted by the Audit Report and the lack of progress in implementing a comprehensive and accurate electronic database for the Department. Despite DVA’s reassurances, the Committee is concerned at the potential for a detrimental flow-on effect with regard to client payments and service delivery.

4.45               The Committee recognises the importance of DVA to veterans and stresses the need for the Department to continue to adapt to the needs of its changing client base and deliver the same high level of support as it has in the past. To facilitate delivery, the Committee urges DVA to implement the ANAO recommendations to improve the data integrity of its income support records whichever system is selected after the current review of its IT services.